With the development of digital technology in recent years, the transfer rate for digitizing an image and outputting it as digital image data reaches several hundreds of megabits/sec.about.several gigabits/sec. When transmitting or recording the digital image data, the transmission is restricted by the communication cost and the recording is restricted by the recording capacity. So, there are proposed a coding technique of compressing digital image data to obtain coded image data and a decoding technique of decompressing the coded image data to obtain the original image, whereby compressed digital data are transmitted and recorded.
FIG. 8 is a diagram for explaining the compressive coding and the decompressive decoding. As shown in FIG. 8, generally, image data, that is, an array of digitized pixels, is divided into blocks each having a prescribed size (8.times.8 pixels in FIG. 8). Each pixel has a pixel value relating to its color and luminance. In this stage, however, the pixel values are in the random state as the original image. When these pixels are subjected to transform such as DCT (Discrete Cosine Transform), pixels of large pixel values are biased as shown in the figure. So, by quantization, each term is arithmetically divided by a prescribed number and terms of less contribution are treated as 0, thereby compressing the image data. Thereafter, variable-length code conversion is sometimes carried out. As shown in FIG. 8, when the coded data is transmitted and received, the coded data is restored to the original image in a decoding process comprising inverse quantization, i.e., inverse arithmetic of the above-mentioned quantization, and inverse transform, i.e., inverse arithmetic of the above-mentioned transform, whereby an image approximately equal to the original image is obtained.
Further, variable-length coding, such as run-length coding, enables further compression of data. For example, in run-length coding, data compression is achieved as follows. That is, the number of continuous pixels having the same pixel values is coded as a 2-byte code (run) and the pixel value is coded as a 1-byte code (length). Hereinafter, a notation like [00] means 1 byte in hexadecimal numeral.
For example, assuming that an array of pixel values, [00][00][00][01][01][02][02][02][02][02], is given, this is coded as follows. Since three pieces of [00] continue from the head of the array, 2-byte data [00][03] showing `three pieces` and 1-byte data [00] showing the pixel value are given. Subsequently, since two pieces of [01] continue, 2-byte data [00][02] showing `two pieces` and 1-byte data [01] showing the pixel value are given. Finally, for subsequent five pieces of [02], 2-byte data [00][05] showing `five pieces` and 1-byte data [02] showing the pixel value are given. These data are arranged as [00][03][00][00][02][01][00][05][02], resulting in coded data compressed by 10%. As the number of continuous pixels of the same pixel value increases, the compression ratio increases.
Furthermore, coding is generally carried out on the basis of the spatial or temporal local correlation possessed by image, and this coding is called "local coding". There are two kinds of local coding processes: intra-frame coding that eliminates redundancy according to the spatial correlations of pixels in a frame, and inter-frame coding that eliminates redundancy according to temporal correlations between different frames.
FIGS. 9(a) and 9(b) are diagrams for explaining the intra-frame coding and the inter-frame coding, respectively. As shown in FIG. 9(a), in a still picture, there is a strong possibility that adjacent pixels will have the same or near pixel values. The intra-coding employs this correlation for compression, according to the conception of space frequency.
On the other hand, as shown in FIG. 9(b), a moving picture is digitized and treated as plural pieces of still pictures. There is a strong possibility that, among the plural still pictures, adjacent still pictures like N-1 and N will be identical or similar, and the inter-frame coding is carried out on the basis of this fact. For example, a difference between digital image data N-1 and N is obtained, and only a portion of N different from N-1 is subjected to the subsequent processing, thereby to compress the image data. Even in the inter-frame coding, the first still picture is processed by the intra-frame coding.
In the above-mentioned compressive coding, to increase the compression ratio results in degraded image quality. So, in order to achieve appropriate compression, "rate control" for controlling the coded data generation rate is required. This rate control is performed by appropriately setting the quantization scale according to the input data because the quantization scale, which is used for division in the above-mentioned quantization process, is directly linked with the information quantity generated as coded data.
An image coding apparatus performing such rate control according to a prior art (ISO-IEC/JIC1/SC29/WG11 NO328) will be described hereinafter.
FIG. 7 is a block diagram illustrating the image coding apparatus. As shown in the figure, the apparatus comprises a subtracter 1, an image transformer 2, a quantizer 3, a variable-length encoder 4, an inverse quantizer 5, an inverse image transformer 6, an adder 7, an image storage unit 8, an activity calculator 22, and a quantization scale decision unit 23.
The subtracter 1 calculates a difference between an input image per coding unit and a reference image created from an already coded image, providing an input image to be transformed. The image transformer 2 receives the input image to be transformed, and performs prescribed transformation to the image, thereby generating a transformed image. The quantizer 3 quantizes the transformed image according to a modified quantization scale from the quantization scale decision unit 23, thereby generating a transformed and quantized coefficient. The variable-length encoder 4 receives the transformed and quantized coefficient and performs variable-length coding, thereby generating a coding result to which the modified quantization scale is added. Further, the encoder 4 measures the information quantity of the coding result and outputs it to the quantization scale decision unit 23. The inverse quantizer 5 receives the transformed and quantized coefficient and the modified quantization scale, and generates inversely quantized image. The inverse image transformer 6 receives the inversely quantized image and performs prescribed inverse transformation, thereby generating an inversely transformed image. The adder 7 adds the reference image and the inversely transformed image to generate a locally decoded image. The image storage unit 8 stores the locally decoded image for a prescribed period of time, and outputs the reference image.
The activity calculator 22 calculates an activity and an average activity which are used for performing rate control with a modified quantization scale. The quantization scale decision unit 23 decides a modified quantization scale on the basis of the activity and the average activity calculated by the activity calculator 22, the information quantity of the coding result output from the variable-length encoder 4, and a given target bit number. The modified quantization scale is input to the quantizer 3, the inverse quantizer 5, and the variable-length encoder 4.
A description is given of the operation of the image coding apparatus shown in FIG. 7, with respect to [A. Coding], [B. Local Decoding], and [C. Rate Control].
[A. Coding]
Initially, an image is divided into plural sub-areas according to digitized data of the image, and each sub-area image is successively input to the apparatus as an input image S100 per coding unit. The input image is composed of four luminance signal blocks and two color-difference signal blocks, and each block comprises 8.times.8 pixels. This block set is called a macroblock. The image S100 per coding unit (macroblock) is coded by the intra-frame coding or the inter-frame coding based on a prescribed criterion. In this example, only the first macroblock is coded by intra-frame coding, and subsequent macroblocks are coded by the inter-frame coding.
In a local decoding process described later, a reference image S108 is generated using a locally decoded image S107 stored in the image storage unit 8, and an input image S101 to be transformed is generated from the image S100 and the reference image S108 by the subtracter 1. The input image S101 is transformed in the image transformer 2 to be a transformed image S102. The transformed image S102 is quantized by the quantizer 3 to be a transformed and quantized coefficient S103. The quantization scale used in the quantization is decided for each coding target (macroblock). An initial value set in advance is used at the beginning and, when a modified quantization scale is given by the quantization scale decision unit 23, the modified quantization scale is used thereafter.
Further, the variable-length encoder 4 performs run-length coding to the transformed and quantized coefficient S103 in a prescribed order from the lower band toward the higher band, and outputs a bitstream S104 which is the coding result of the image S100. Further, for rate control, the variable-length encoder 4 measures the information quantity of the coding result as bit number b and outputs this toward the quantization scale decision unit 23.
[B. Local Decoding]
As described above, in the inter-frame coding, a difference image is obtained using a previously processed image as a reference image S108, so that it is necessary to locally decode the coded data so that the decoding result can be used as the reference image.
The transformed and quantized coefficient S103 obtained by the quantizer is also input to the inverse quantizer 5. Then, the inverse quantizer 5 performs inverse quantization to generate an inversely quantized image S105. The inversely quantized image S105 is inversely transformed by the inverse image transformer 6 to be an inversely transformed image S106. The inversely transformed image S106 is added to the reference image by the adder 7, thereby generating a locally decoded image S107. The locally decoded image S107 is stored in the image storage unit 8 for a prescribed period of time, whereby a reference image S108 for the next input is obtained.
[C. Rate Control]
Rate control by modifying the quantization scale is carried out as follows. The activity calculator 22 calculates an activity act per input image S100, and outputs the activity to the quantization scale decision unit 23. To be specific, the activity is obtained by calculating variances of the respective blocks of luminance signal in the image S100 and adding 1 to the smallest variance. Further, the activity calculator 22 obtains an average of activities of the input image S100 within the frame, and outputs an average activity avg.sub.-- act toward the quantization scale decision unit 23. The average activity avg.sub.-- act input to the quantization scale decision unit 23 is a value calculated with respect to a frame for which coding has just been completed.
The quantization scale decision unit 23 decides a modified quantization scale, according to a target bit number Tb of coding result generated in a frame obtained by a prescribed method, the bit number b obtained as the information quantity of the above-mentioned coding result, the activity act of the image S100, and the average activity avg.sub.-- act.
The quantization scale decision unit 23 decides the modified quantization scale in the following manner. The quantization scale decision unit 23 retains parameters of buffer fullness dj (quantity of data in a virtual buffer) corresponding to the respective coding modes for coding the original image, i.e., intra-frame coding, inter-frame coding using forward prediction, and inter-frame coding using bidirectional prediction, and it performs rate control based on the buffer fullness dj. The buffer fullness dj is updated to a value obtained in formula (1) when coding of the image S100 has been completed. ##EQU1## where d0 is a value of buffer fullness dj when processing of a previous macroblock has just been completed, and MBn is the number of macroblocks constituting the frame of the original image.
Using the buffer fullness dj, a quantization scale qj is obtained in formula (2). ##EQU2## where r is the reaction parameter which is represented by formula (3). In formula (3), bit.sub.-- rate is the target bit number of coding result per second, and picture.sub.-- rate is the number of input original images per second. ##EQU3##
Further, the quantization scale qj is subjected to modulation in formula (4), thereby generating a modified quantization scale mg. The modified quantization scale mg is input to the quantizer 3. ##EQU4##
On the basis of the modified quantization scale mg obtained in formula (4), the quantizer 3 performs quantization. In addition, the modified quantization scale mq is input to the inverse quantizer 5 and to the variable-length encoder 4. In the inverse quantizer 5, it is used for inverse quantization. In the variable-length encoder 4, it is coded and incorporated in the coding result. That is, it is incorporated as information to be used for inverse quantization when the coding result is subjected to decoding.
As described above, in the image coding apparatus according to the prior art, the quantizer 3 modifies the quantization scale for quantization by feedback control, thereby to perform rate control. Therefore, the data quantity of the coding result can be approximated to the target data quantity set in advance.
However, the image coding apparatus has the following drawbacks.
In the image coding apparatus, rate control is carried out for the purpose of making the bit number per coding unit (macroblock) uniform by adjusting it to the target bit number. That is, in formula (1), an error between "the bit number b of coding result output for each macroblock" and "the value obtained by equally dividing the target bit number Tb with the number of macroblocks" is accumulated and fed back, thereby to realize rate control. In this case, appropriate rate control is possible as long as the information quantity of the input image to be transformed (S101 in FIG. 7) is uniform over the original image. However, when the information quantity of the input image is considerably biased, rate control doe s not function satisfactorily. Further, because of the rate control, local characteristics of the original image cannot be reflected , resulting in degraded quality of coded image.